How to connect via Session Manager?
Minimal steps to connect
SSM Agent
Prepare Instance/AMI with installed SSM Agent or use one from this list
Manually installing SSM Agent on EC2 instances for Linux
Example for Centos Stream:
#Global resource:
sudo dnf install -y https://s3.amazonaws.com/ec2-downloads-windows/SSMAgent/latest/linux_amd64/amazon-ssm-agent.rpm
#Specific region: us-east-1
sudo dnf install -y https://s3.us-east-1.amazonaws.com/amazon-ssm-us-east-1/latest/linux_amd64/amazon-ssm-agent.rpm
sudo systemctl status amazon-ssm-agent
sudo systemctl enable amazon-ssm-agent
sudo systemctl start amazon-ssm-agent
Example for Ubuntu 20.10:
sudo snap install amazon-ssm-agent --classic
sudo snap list amazon-ssm-agent
sudo snap start amazon-ssm-agent
sudo snap services amazon-ssm-agent
Read more:
IAM instance profile
Create an IAM instance profile for Systems Manager and attach IAM role to instance (https://docs.aws.amazon.com/systems-manager/latest/userguide/setup-instance-profile.html)
Create instance profile:
Important:
- When you create roles for EC2, profile instances are automatically created.
- If you create roles via AWS CLI the profile instance is not created. You have to create it manually.
What does it look like with the AWS CLI?
You need a trust policy for the role
cat <<EOT >> Role-Trust-Policy.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
EOT
Next you need to create a role with necessary policies, instance profile and attach role to instance profile:
aws iam create-role --role-name Role-SSM-instance --assume-role-policy-document file://Role-Trust-Policy.json
aws iam attach-role-policy --policy-arn arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore --role-name Role-SSM-instance
aws iam create-instance-profile --instance-profile-name SSM
aws iam add-role-to-instance-profile --role-name Role-SSM-instance --instance-profile-name Role-SSM-instance
You can see the result:
aws iam list-instance-profiles
Now we can associate IAM instance profile to the instance:
aws ec2 associate-iam-instance-profile --instance-id INSTANCE_ID --iam-instance-profile Name=SSM
Read more:
- IAM roles and instance profile https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html
- Instance Profile https://docs.aws.amazon.com/cli/latest/reference/iam/create-instance-profile.html
Allow HTTPS traffic in Security Group
Security group added to VPC endpoint must allow inbound HTTPS (port 443) traffic from the resources in your VPC that communicate with the service. In SG for EC2, we do not need to allow HTTPS traffic just to set up a connection with Session Manager.
The security group must allow inbound HTTPS (port 443) traffic from the resources in your VPC that communicate with the service.
This is an example for AWS CLI:
aws ec2 authorize-security-group-ingress \
--group-id sg-ID-Security-Group \
--protocol tcp \
--port 443 \
--source-group sg-1a2b3c4d
Read more:
Connecting to instance
AWS Management Console
To start a session (Amazon EC2 console)
- Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.
- In the navigation pane, choose Instances.
- Select the instance and choose Connect.
- For Connection method, choose Session Manager.
- Choose Connect.
Read more:
- Start a session: https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-working-with-sessions-start.html
AWS Command Line Interface
You need to use this command from your local instance or CloudShell
aws ssm start-session --target instance-id
Read more:
- Start a session: https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-working-with-sessions-starthtml#sessions-start-cli
- CloudShell https://aws.amazon.com/cloudshell/
Enable session logging to S3 and CloudWatch with encryption
You can log session commands and details in an Amazon S3 bucket or CloudWatch Logs log group. Configure sessions on the Session Manager Preferences page for CloudWatch logging or S3 logging.
Important:
-
Each command together with the output is logged.
-
Logs are only collected for Session Manager connections.
-
The log stream is separate for each session in CloudWatch and have the time when this command executed.
-
The logs are recorded in real time in CloudWatch.
-
In S3 we can see separate files for each session.
-
Logs are recorded after the session ends on S3.
-
On S3 We do not have the time when the command was executed, only the start and end of the session.
Read more:
- Logging session activity https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-logging.html
What do we need to do?
- General
- Create a KMS key for encryption and set up the key policy to grant permission to the IAM role that will be used for session logging.
- Update an IAM policy that allows access to the S3 bucket and CloudWatch Logs for session logging.
- S3
- Create an S3 bucket for storing session logs with encryption.
- In the Session Manager settings, enable "Send session logs to S3" and "Choose a bucket name from the list". You can Enforce encryption.
- CloudWatch
- Create a CloudWatch Log Group for storing session logs with encryption.
- Update an IAM policy that allows access to the S3 bucket and CloudWatch Logs for session logging and using KMS key.
- In the Session Manager settings, enable "CloudWatch logging", choose "Stream session logs" and "Choose a log group name from the list". You can Enforce encryption.
Additional ways to configure Session Manager Preferences:
- AWS CLI
{
"schemaVersion": "1.0",
"description": "Document to hold regional settings for Session Manager",
"sessionType": "Standard_Stream",
"inputs": {
"s3BucketName": "bucket-s3-logs-sessionmanager",
"s3KeyPrefix": "ec2session",
"s3EncryptionEnabled": true,
"cloudWatchLogGroupName": "EC2Session",
"cloudWatchEncryptionEnabled": true,
"cloudWatchStreamingEnabled": true,
"kmsKeyId": "",
"runAsEnabled": false,
"runAsDefaultUser": "",
"idleSessionTimeout": "",
"maxSessionDuration": "",
"shellProfile": {
"windows": "",
"linux": ""
}
}
}
aws ssm update-document --name "SSM-SessionManagerRunShell" --content "file://SessionManagerRunShell.json" --document-version "\$LATEST"
-
CloudFormation with CustomResources. Preparing Lambda function and run this. (included configuration in the CloudFormation template). If you want automated this, you can use this solution. I added to CloudFormation template at the end this file. Brief information: During CF create, there are running another stack with EC2 instance. To EC2 instnane you send the command. You can put Log group name to create and collect logs from this instance. This is usefull during occurs the problems. You need to prepare IAM role/ Instance Profile for this EC2 with necessary permission to run the command. Unfortuantely this take more time to finish the CF stack.
-
Using AWSUtility::CloudFormation::CommandRunner. This creates an instance EC2 to run the command.
Read more:
- AWSUtility::CloudFormation::CommandRunner https://aws.amazon.com/premiumsupport/knowledge-center/cloudformation-commandrunner-stack/
- https://docs.aws.amazon.com/systems-manager/latest/userguide/getting-started-configure-preferences-cli.html
Warnning:
In this case, there is a potential risk of data leakage. Imagine setting up a configuration under SSM Prefernace and collecting logs in S3 without forcing encryption. We resigned from the solution and we deleted the S3. External account create the bucket S3 with the same name what in our configuration. What is happening?
Our logs are saved to this bucket! This is our all session. We can open some confidential file or all share our configuration.
If we use "Enforce encryption" we can get this error: Your session has been terminated for the following reasons: Couldn't start the session because we are unable to validate encryption on Amazon S3 bucket. Error: AccessDenied: Access Denied status code: 403
If not we can put session file to bucket, where permission are appropriate.
Troubleshooting
- One of the less transparent cases that can occur:
Your session has been terminated for the following reasons: We couldn't start the session because encryption is not set up on the selected CloudWatch Logs log group. Either encrypt the log group or choose an option to enable logging without encryption.
and a sample solution:
Add a policy to the IAM instance profile that is attached to your instance granting permission to upload encrypted logs to CloudWatch
Read more:
- CloudWatch https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/encrypt-log-data-kms.html
- Create a custom IAM role for Session Manager https://docs.aws.amazon.com/systems-manager/latest/userguide/getting-started-create-iam-instance-profile.html#create-iam-instance-profile-ssn-logging
- Troubleshoot https://aws.amazon.com/premiumsupport/knowledge-center/ssm-session-manager-failures/